Establishing meaningful cut points for online user ratings Establishing meaningful cut points for online user ratingsEstablishing meaningful cut points for online user ratings

Subjective perceptions of websites can be reliably measured with questionnaires. But it is unclear how such scores should be interpreted in practice, e.g. is an aesthetics score of 4 points on a 7-point-scale satisfactory? The current paper introduces a ROC-based methodology to establish meaningful cut points for the VisAWI (Visual Aesthetics of Websites Inventory) and it’s short form the VisAWI-S. In two studies we use users’ global ratings (UGRs) and website rankings as anchors. A total of 972 participants took part in the studies and yielded similar results. First, one-item UGRs correlate highly with the VisAWI. Second, cut points on the VisAWI reliably differentiate between sites that are perceived as attractive vs. unattractive. Third, these cut points are variable, but only within a certain range. Together the research presented here establishes a score of 4.5 on the VisAWI is a reasonable goal for website designers and highlight the utility of the ROC methodology to derive relevant scores for rating scales.